Electronic product recognition

ABSTRACT

Methods and systems for identifying one or more products in an electronic image are disclosed. The computer-implemented method to electronically recognize a product in an electronic image captured via an electronic mobile device is disclosed. The method includes receiving, by a server, a video stream from a camera of the electronic mobile device, the video stream including a plurality of frames. The server selects at least one of the plurality of frames from the video stream, the at least one of the plurality of frames from the video stream being selected is a captured image. The server segments a plurality of products in the captured image into a plurality of segments. The server performs an image recognition using each of the plurality of segments to identify the product in each of the plurality of segments. One or more recognized products identified in the image recognition is outputted by the server.

FIELD

This disclosure relates generally to electronic image recognition. Morespecifically, this disclosure relates to electronic image recognition ofan image including one or more products available for purchase from aretailer using an electronic device in a retail environment.

BACKGROUND

Retail shopping continues to evolve as electronic mobile devices developadditional functionality. Consumers can now use a camera on theelectronic mobile device to, for example, scan a barcode on a product tolearn more information about that product. Augmented reality can beutilized to superimpose information onto a display of the consumer'selectronic mobile device to enable virtual interactions with thereal-world environment.

Improved ways of identifying the products captured from the consumer'selectronic device are desirable.

SUMMARY

This disclosure relates generally to electronic image recognition. Morespecifically, this disclosure relates to electronic image recognition ofan image including one or more products available for purchase from aretailer using an electronic device in a retail environment.

In an embodiment, a product can be recognized within at or about twoseconds and with confidence of at or about 95% or greater.

In an embodiment, the electronic product recognition can be performed inreal-time or substantially real-time.

A system is disclosed. The system includes a server having a processorand a memory; a communication network; and a database in electroniccommunication with the server via the communication network. The serverincludes an electronic product recognizer that receives a video streamfrom a camera of an electronic mobile device, the video stream includinga plurality of frames. The server selects at least one of the pluralityof frames from the video stream, the at least one of the plurality offrames from the video stream being selected is a captured image. Theserver segments a plurality of products in the electronic image into aplurality of segments. The server performs an image recognition usingeach of the plurality of segments to identify the product in each of theplurality of segments. The server outputs one or more recognizedproducts identified in the image recognition.

A computer-implemented method to electronically recognize a product inan electronic image captured via an electronic mobile device isdisclosed. The method includes receiving, by a server, a video streamfrom a camera of the electronic mobile device, the video streamincluding a plurality of frames. The server selects at least one of theplurality of frames from the video stream, the at least one of theplurality of frames from the video stream being selected is a capturedimage. The server segments a plurality of products in the captured imageinto a plurality of segments. The server performs an image recognitionusing each of the plurality of segments to identify the product in eachof the plurality of segments. The server outputs one or more recognizedproducts identified in the image recognition.

A system is disclosed. The system includes a server having a processorand a memory; a communication network; and a database in electroniccommunication with the server via the communication network. The serverincludes an electronic product recognizer that receives a segmentedimage from a camera of an electronic mobile device, the segmented imageincluding a plurality of segments. The server performs an imagerecognition using each of the plurality of segments to identify theproduct in each of the plurality of segments. The server outputs one ormore recognized products identified in the image recognition.

A computer-implemented method to electronically recognize a product inan electronic image captured via an electronic mobile device isdisclosed. The method includes receiving, by a server, a segmented imagefrom a camera of the electronic mobile device, the video streamincluding a plurality of segments. The method further includesperforming an image recognition using each of the plurality of segmentsto identify the product in each of the plurality of segments. The serveroutputs one or more recognized products identified in the imagerecognition.

BRIEF DESCRIPTION OF THE DRAWINGS

References are made to the accompanying drawings that form a part ofthis disclosure and which illustrate embodiments in which the systemsand methods described in this specification can be practiced.

FIG. 1 is a schematic diagram of a system for implementing theelectronic product recognition systems and methods described in thisspecification, according to an embodiment.

FIG. 2A is a flowchart of a method for recognizing a plurality ofproducts within a captured image, according to an embodiment.

FIG. 2B is a flowchart of a method for recognizing a plurality ofproducts within a captured image, according to an embodiment.

FIG. 3 is a schematic diagram of a captured image that has beensegmented according to the method in FIG. 2A or FIG. 2B, according to anembodiment.

FIG. 4 is a flowchart of a method for performing image recognition,according to an embodiment.

FIG. 5 is a schematic diagram of architecture for a computer device,according to an embodiment.

Like reference numbers represent like parts throughout.

DETAILED DESCRIPTION

This disclosure relates generally to electronic image recognition. Morespecifically, this disclosure relates to electronic image recognition ofan image including one or more products available for purchase from aretailer using an electronic device in a retail environment.

Augmented reality generally refers to technology in which acomputer-generated image is superimposed on a display device of aconsumer's electronic device (e.g., an electronic mobile device, etc.)to provide a composite image that includes the consumer's view of thereal world as well as the superimposed image. In today's retailenvironment, augmented reality is being utilized to superimposeinformation about one or more products sold in a retailer's store toprovide the consumer with additional information about the products.Success of the augmented reality application is based at least in parton the ability of the retailer to identify what products are in theconsumer's view of the real world (through the electronic mobiledevice).

Image recognition currently may be limited to recognizing a single typeof object at a time. For example, current image recognition typically isapplied in recognizing a single product. However, retail stores may havethousands of products arranged on a shelf including a number of similarproducts. For a consumer to utilize current image recognition, theconsumer must position the product and wait for the image recognition tocomplete. This can be slow and inconvenient for the consumer, especiallyin instances where the consumer would like to recognize more than oneproduct. Improved methods of recognizing a plurality of productsaccurately and quickly are desirable.

In an embodiment, a retailer can limit a scope of the image recognitionbased on the products that the retailer sells. Limiting the scope of theimage recognition can reduce an overall computational load in completingthe image recognition and increase a speed at which the imagerecognition is completed.

Disclosed herein are electronic product recognition methods and systemsin which an electronic image or images are received from an electronicmobile device and the electronic images are utilized to identify theproduct or products included in the electronic image and provide productinformation relevant to the product or products identified in theelectronic image(s).

An electronic product recognizer can receive the electronic images froma camera on an electronic mobile device and generate a list ofrecognized products. The list of recognized products permits theconsumer to interact with the recognized products via the mobile device.

As described in further detail below, in one embodiment, a consumer usesa camera on an electronic mobile device, such as a smartphone, tabletdevice, wearable device, or the like, to capture an electronic image ofa product or a plurality of products using an application that resideson the electronic mobile device. The product or products can then berecognized. A list of products as recognized (e.g., recognized products)can be generated and a sticker or other window may be displayed on thedisplay screen of the electronic mobile device to enable the consumerto, for example, learn more about the products, shop the products and,if desired, purchase one or more of the products. The list of productsfrom the image may generally be referred to as recognized productsherein.

The product or products can be any type of product that the retailersells. The products are products that are available for sale within aretail store. The products may be available for sale online (e.g., on awebsite of the retail store, etc.) or that are available for sale viaanother sales channel. In an embodiment, products in the electronicimages that are not for sale may be excluded from the image recognition.For example, products for promotional or other purposes that areintended to permit consumers to learn more about a product, but notnecessarily to permit the consumer to be able to purchase the product,may be excluded from the image recognition.

As used herein, an electronic mobile device is any device that can beused to electronically capture an image and that can display a list ofrecognized products on a display screen that is connected to orassociated with the electronic mobile device.

Examples of electronic mobile devices include, but are not limited to,mobile phones, smartphones, tablet-style devices, wearable devices,laptop computers, and the like. In one embodiment, the electronic mobiledevice includes at least a camera and some means to control the camera.The electronic mobile device also includes a display screen and somemeans to select products displayed on the display screen, and some meansto send and receive data communications as described further below.

Cameras on most current electronic mobile devices are capable ofcapturing photographs and/or video. In an embodiment, the camera may becapable of capturing an image without the user performing an action(e.g., pressing a shutter button) to cause an image to be taken. Anytype of image capture technology on an electronic mobile device that iscapable of capturing an image of one or more products can be used.

As used herein, to capture or capturing an image refers to the act ofobtaining an image of the products using the camera or other imagecapture technology of the electronic mobile device. Obtaining the imageof the products may also be referred to as imaging the products. Acaptured image is an image of the products that has been captured by theelectronic mobile device. The products from which a captured image hasbeen obtained may be referred to as the imaged products. It is to beappreciated that capturing an image of the products includes capturingan image of the entire product as well as capturing an image of aportion of the product.

As used herein, a recognized product, or a list of recognized productsincludes those products identified from a captured image.

As used herein, optical character recognition (OCR) includes anelectronic conversion of an image of text into machine-readable text.

As used herein, histogram of oriented gradients includes a technique inwhich occurrences of gradient orientation in localized portions of animage are counted.

As used herein, edge detection includes identifying locations within anelectronic image in which image brightness abruptly changes.

As used herein, a marker includes an electronic visual cue whichtriggers display of electronic information.

As used herein, a planogram includes an electronic representation ofproducts within a retail store.

In an embodiment, a product can be recognized within at or about twoseconds and with confidence of at or about 95% or greater.

In an embodiment, the product recognition occurs in real-time orsubstantially real-time.

In an embodiment, the one or more products can be recognized regardlessof location. For example, the products can be arranged on a shelf, in acart, in a consumer's hand, or the like.

FIG. 1 is a schematic diagram of a system 10 for implementing theelectronic product recognition systems and methods described in thisspecification, according to an embodiment. The system 10 can identifyone or more products captured in a real world scene from a user device15. In an embodiment, the system 10 can be utilized in a retail storeenvironment to enable a consumer to capture an image of one or moreproducts located in the retail store while, for example, the consumer isbrowsing the retail store.

The system 10 includes a server 25 in electronic communication with aplurality of user devices 15 via a network 20. The server 25 includes anelectronic product recognizer 60 that can receive a captured image fromthe user devices 15. The electronic product recognizer 60 can identifyone or more products included in the captured image received from theuser devices 15. The electronic product recognizer 60 can make theproduct information available to the system 10, as described in thisSpecification available, to the user devices 15 via the network 20. Inan embodiment, the electronic product recognizer 60 can be configured togenerate one or more messages to a user that includes the productinformation or a message with details about the product.

The electronic product recognizer 60 can include a plurality of modulesutilized in the identification of the products within the capturedimage. In the illustrated embodiment, three modules 65-75 are shown. Itwill be appreciated that the number of modules can vary within the scopeof this disclosure.

In the embodiment shown in FIG. 1, the electronic product recognizer 60includes a marker detector 65, an optical character recognizer 70, and alogo recognizer 75. In an embodiment, the one or more additional modulescan include an edge detector, a planogram identifier, a histogram oforiented gradient detector, a global positioning system (GPS)recognizer, an in-store positioning system (IPS), a color recognizer, anartificial neural network, or the like.

The marker detector 65 can identify one or more known markers in acaptured image. A marker generally includes an electronic visual cuewhich triggers display of electronic information. For example, a markercan include a barcode on a shelf, a tag or other aspect of a displaywhich is included in the captured image, an aspect of product packaging,or the like.

The optical character recognizer 70 can identify one or more textcharacters in the captured image. Optical character recognition (OCR)systems are generally well-known and the optical character recognizer 70functions according to these generally known principles.

The logo recognizer 75 can identify a logo or brand identifier presenton the one or more products included in the captured image.

In an embodiment, the network 20 may be representative of the Internet.In an embodiment, the network 20 can include a local area network (LAN),a wide area network (WAN), a wireless network, a cellular data network,suitable combinations thereof, or the like. Aspects of the network 20can be the same as or similar to aspects of the network 540 as shown anddescribed in accordance with FIG. 5 below. In an embodiment, the userdevices 15 may be located on premises of a retail store of the retailerand the server 25 and database 30 located on premises as well. In anembodiment, one or more components of the server 25 or the database 30may be located off premises.

Examples of the user devices 15 include, but are not limited to, amobile device (e.g., a smartphone, a personal digital assistant (PDA), atablet-style device, etc.), a wearable mobile device (e.g., a smartwatch, a head wearable device, etc.), or the like. The user devices 15generally include a display device and an input device. Examples of thedisplay devices for the user devices 15 include, but are not limited to,a mobile device screen, a tablet screen, a wearable mobile devicescreen, or the like. Examples of the input devices for the user devices15 include, but are not limited to, a keyboard, a button, a voicecommand, a proximity sensor, a touch sensor, an ocular sensing devicefor determining an input based on eye movements (e.g., scrolling basedon an eye movement), suitable combinations thereof, or the like. Aspectsof the user devices 15 can be the same as or similar to aspects of theuser devices 501 as shown and described in accordance with FIG. 5 below.

The server 25 is in electronic communication with database 30. Thedatabase 30 can include a variety of data related to the products beingrecognized by the electronic product recognizer 60.

In an embodiment, the database 30 can include one or more databases. Forexample the database 30 shown in FIG. 1 includes a product database 35and a marker database 40.

The product database 35 can generally store information related to allof the products available from the retailer. In an embodiment in whichthe server 25 and the database 30 are stored on premises of one of theretailer's stores, the product database 35 can include information thatis unique to the particular store. For example, a first retail store ofthe retailer may sell 10,000 products and a second retail store may sell10,000 products where at least one of the products sold at the firstretail store is not available at the second retail store, or vice versa.In such a scenario, the product database 35 on premises of the firstretail store may include different information than the product database35 on premises of the second retail store. It will be appreciated thatin an embodiment, even if the retailer sells different products atdifferent stores, the product database 35 at each of the retail storesmay include the same information (including items not sold at aparticular store) for simplicity of managing the data in the productdatabase 35.

The product database 35 may be connected to an external network so thatupdates can be made on a periodic basis to the information stored in theproduct database 35. For example, on a daily, weekly, etc., basis, theproduct database 35 may be updated to remove product information that isassociated with a product the retailer no longer sells, to add productinformation for a new product that the retailer is beginning to sell, tomodify information about products the retailer currently sells, or thelike. It will be appreciated that updating the product database 35 canalso be updated as needed (e.g., on an irregular basis).

In general, the product database 35 may include any information that isdescriptive of the product. For example, the product database 35 caninclude a unique identifier for the product, the product name, producttype, price, etc. Additionally, the product database 35 can includeinformation relevant to the identification of the product in a capturedimage. For example, the product database 35 can include colorinformation, label text, product location in the store, nearby products,or the like.

The marker database 40 includes markers associated with productsavailable in a retailer's store.

It will be appreciated that the above databases are examples. Additionaldatabases can be included. One or more of the databases can be combinedinto a single database. For example, in an embodiment, the productdatabase 35 and the marker database 40 can be combined into a singledatabase (not shown).

It is to be appreciated that various roles of user devices 15, server25, and database 30 may be distributed among the devices in the system10. For example, the electronic product recognizer 60 can be partiallyor entirely included on the user devices 15 in, for example, anapplication or the like. Similarly, the database 30 can be maintained onthe server 25.

In general, the system 10 is disposed physically at the retail store ofthe retailer. That is, eliminating communication between the userdevices 15, server 25, and database 30 occurring via a network remotefrom the retail store may increase a performance and response time ofthe system 10. In an embodiment, the server 25 and database 30 may bephysically located at the retail store, but may be connected incommunication with an external network so that, for example, updates maybe provided on an ongoing basis (e.g., nightly, weekly, etc.).

FIGS. 2A and 2B illustrate methods for recognizing a plurality ofproducts within a captured image, according to an embodiment.

FIG. 2A is a flowchart of a method 100 for recognizing a plurality ofproducts within a captured image, according to an embodiment. The method100 generally includes receiving a captured image from a user device anddetermining what products are visible in the captured image. Thedetermination of what products are visible in the captured image can beperformed with greater than 95% confidence. In an embodiment, thedetermination can be performed in less than at or about two seconds. Itwill be appreciated that the timing is an example, and the actual targetfor timing can vary beyond this number. If the method 100 does notidentify the products visible in the captured image with greater than95% confidence, the method 100 may not provide recognized products tothe user devices 15. In an embodiment, if the method 100 does notidentify the products visible in the captured image with greater than95% confidence, the server may initiate a message to the user devicethat enables the user to identify the product in the captured image.

At 105, a server (e.g., the server 25 in FIG. 1) receives a video streamfrom a user device (e.g., the user devices 15 in FIG. 1). The videostream may include a plurality of still images (frames) played insuccession. In general, the electronic mobile device may have a framerate of at or about 20 to at or about 30 frames per second (FPS). Itwill be appreciated that these frame rates are examples and can varyaccording to, for example, the particular electronic mobile device beingused. The frame rate of the electronic mobile device may determine howmay frames are provided to the server 25.

At 110, the server 25 selects a frame of the video stream. The frameserves as the captured image. In this manner, the consumer is able toview the one or more products via the camera device on the electronicmobile device without taking action to capture the image. Also at 110,the server segments the products included in the captured image (e.g.,the selected frame) into discrete product images. The segmenting caninclude, for example, breaking the captured image into a series ofdiscrete segments. In an embodiment, the segments can be, for example,rectangles or the like. In an embodiment, the selected frame can includea single frame from the video stream. In an embodiment, the selectedframe can include a plurality of frames from the video stream.

An example of a captured image that has been segmented is shown in FIG.3. In FIG. 3, the captured image 150 shows a plurality of bottles 155 ona plurality of shelves 160. Each of the bottles 155 is shown with arectangle (segment) 165 drawn around the bottles 155. Referring back toFIG. 2, the segmenting at 110 can be performed using various approachesto complete the segmenting. At 110, the server 25 may use an opticalcharacter recognition (OCR), an edge detection, a histogram of orientedgradients, a marker, an artificial neural network, or combinationsthereof, to identify where the segments 165 should be placed. Followingthe segmentation at 110, image recognition can be performed on each ofthe plurality of segments 165 (e.g., different product images) toidentify which product is in each segment.

At 115 the server determines matching products for each of the segmentedimages for the plurality of products in the captured image using theelectronic product recognizer 60. In an embodiment, one manner ofincreasing the performance of the electronic product recognizer 60 is tolimit the pool of possible matching products to those products sold bythe retailer (e.g., only those products identified in the productdatabase 35). Accordingly, if the consumer is imaging a product orproducts that are not for sale by the retailer, then the productrecognizer 50 will not identify the product or products as a recognizedproduct.

Additionally, 115 is performed for each of the segmented images from110. The recognition for the various images can be performedconcurrently to meet the processing time requirements of identifyingproducts in less than 2 seconds. Furthermore, as will be described inaccordance with FIG. 4 below, 115 includes a plurality of recognitionprocesses that are concurrently executed. As a result, the recognitionprocess may be a “race to the finish” in which the first recognitionprocess to identify the product with greater than 95% confidence may beused as the recognized product.

At 120, after the recognition of the matching products in the capturedimage, product information from the product database 35 can be output bythe server 25 to the user device 15.

FIG. 2B is a flowchart of a method 130 for recognizing a plurality ofproducts within a captured image, according to an embodiment.

The method 130 generally includes receiving a segmented image from auser device and determining what products are visible in the segmentedimage. The determination of what products are visible in the capturedimage can be performed with greater than 95% confidence. In anembodiment, the determination can be performed in less than at or abouttwo seconds. It will be appreciated that the timing is an example, andthe actual target for timing can vary beyond this number. If the method130 does not identify the products visible in the captured image withgreater than 95% confidence, the method 130 may not provide recognizedproducts to the user devices 15. In an embodiment, if the method 130does not identify the products visible in the captured image withgreater than 95% confidence, the server may initiate a message to theuser device that enables the user to identify the product in thecaptured image.

At 135, a server (e.g., the server 25 in FIG. 1) receives a segmentedimage of a selected frame of a video stream from a user device (e.g.,the user devices 15 in FIG. 1). In an embodiment, the segmenting isperformed on the user device 15. As a result, the method 130 may consumeless transmission bandwidth than the method 100 in FIG. 2A. In thismanner, the consumer is able to view the one or more products via thecamera device on the electronic mobile device without taking action tocapture the image. The segmenting can include, for example, breaking thecaptured image into a series of discrete segments. In an embodiment, thesegments can be, for example, rectangles or the like.

An example of a segmented image is shown in FIG. 3. In FIG. 3, thesegmented image 150 shows a plurality of bottles 155 on a plurality ofshelves 160. Each of the bottles 155 is shown with a rectangle (segment)165 drawn around the bottles 155. Referring back to FIG. 2B, thesegmenting can be performed using various approaches to complete thesegmenting such as, but not limited to, an optical character recognition(OCR), an edge detection, a histogram of oriented gradients, a marker,an artificial neural network, or combinations thereof, to identify wherethe segments 165 should be placed. Image recognition can be performed oneach of the plurality of segments 165 (e.g., different product images)to identify which product is in each segment.

At 140 the server determines matching products for each of the segmentedimages for the plurality of products in the segmented image using theelectronic product recognizer 60. In an embodiment, one manner ofincreasing the performance of the electronic product recognizer 60 is tolimit the pool of possible matching products to those products sold bythe retailer (e.g., only those products identified in the productdatabase 35). Accordingly, if the consumer is imaging a product orproducts that are not for sale by the retailer, then the electronicproduct recognizer 60 will not identify the product or products as arecognized product.

Additionally, 140 is performed for each of the segmented images receivedat 135. The recognition for the various images can be performedconcurrently to meet the processing time requirements of identifyingproducts in less than 2 seconds. Furthermore, as will be described inaccordance with FIG. 4 below, 140 includes a plurality of recognitionprocesses that are concurrently executed. As a result, the recognitionprocess may be a “race to the finish” in which the first recognitionprocess to identify the product with greater than 95% confidence may beused as the recognized product.

At 145, after the recognition of the matching products in the capturedimage, product information from the product database 35 can be output bythe server 25 to the user device 15.

FIG. 4 is a flowchart of a method 200 for performing image recognition,according to an embodiment. The method 200 generally corresponds to 115in the method 100 of FIG. 2A or 140 in FIG. 2B. That is, when the method100 is executed, at 115 the method 200 can be performed, and when themethod 130 is executed, at 140, the method 200 can be performed.

The method 200 generally includes a plurality of recognition methodsthat are concurrently performed to result in a recognized product. In anembodiment, the method 200 can be utilized to recognize a product atgreater than 95% confidence. In an embodiment, the determination can beperformed in less than at or about two seconds. It will be appreciatedthat the timing is an example, and the actual target for timing can varybeyond this number. In an embodiment, the method 200 can include one ormore processes that are performed sequentially.

At 205, a captured image of a product that was received by theelectronic product recognizer 60 is filtered for possible matchingproducts using a gross filtering approach. The gross filtering approachcan, for example, reduce a number of possible products to less than 10%of the initial possible products. For example, if the product database35 includes 10,000 products, the gross filtering approach at 205 canreduce a number of possible products to less than 1,000 products. Itwill be appreciated that these numbers are examples, and the number ofpossible products and products following the gross filtering approach at205 can vary beyond the stated values in accordance with the principlesin this disclosure.

The gross filtering approach can rely on a plurality of imagerecognition methods to reduce a number of possible products in thecaptured image. For example, in an embodiment, the gross filteringapproach can include a location determination for the electronic mobiledevice. The location determination can include several considerations.In an embodiment, the location determination can take into accountcoordinates from a global positioning system (GPS) sensor on theelectronic mobile device. The current GPS information can be included inthe determination, as well as GPS information identifying a path bywhich the consumer arrived at the current location. The current GPSinformation, the path of the consumer, and a planogram of the retailstore can be used to make an estimated determination of what theconsumer should have in her field of view. In an embodiment, the GPSinformation can be combined with, or alternatively replaced by, locationinformation from an in-store positioning system (IPS). The grossfiltering can also include a consideration of whether a logo or brandidentifier on the product image can be identified. Utilizing theseapproaches in combination, the possible products can be reduced to amore manageable number of possible products.

At 210, the electronic product recognizer 60 performs a fine filteringon the possible products identified from the gross filtering at 205. Thefine filtering approach can, for example, reduce a number of possibleproducts to less than 2% of the initial possible products. Continuingwith the example above, if the product database 35 includes 10,000products, the fine filtering approach at 210 can reduce a number ofpossible products to less than 200 products. It will be appreciated thatthese numbers are examples, and the number of possible products andproducts following the fine filtering approach at 210 can vary beyondthe stated values in accordance with the principles in this disclosure.

The fine filtering approach can rely on a variety of approaches forreducing a number of possible products. In the fine filtering approach,the electronic product recognizer 60 can additionally utilize a markerassociated with a brand, can determine whether there are any identifyingfeatures on an edge of the shelf (e.g., product identifiers, etc.), acolor of the product, OCR, and combinations thereof. The user path maybe utilized in 210 in addition to 205 to further reduce the possibleproduct images.

At 215 the electronic product recognizer 60 selects a match from thepool of products identified in the fine filtering approach at 210. If amatch cannot be determined from the pool of products identified in thefine filtering approach, then the method 200 may end without selecting amatch. At 215, the determination can be based on the color of theproduct in the image, OCR for text in the image, and markers associatedwith the item, along with suitable combinations thereof. In anembodiment, a tie can be broken randomly. In another embodiment, a tiecan be broken using a selected fine filtering approach.

FIG. 5 is a schematic diagram of architecture for a computer device 500,according to an embodiment. The computer device 500 and any of theindividual components thereof can be used for any of the operationsdescribed in accordance with any of the computer-implemented methodsdescribed herein.

The computer device 500 generally includes a processor 510, memory 520,a network input/output (I/O) 525, storage 530, and an interconnect 550.The computer device 500 can optionally include a user I/O 515, accordingto some embodiments. The computer device 500 can be in communicationwith one or more additional computer devices 500 through a network 540.

The computer device 500 is generally representative of hardware aspectsof a variety of user devices 501 and a server device 535. Theillustrated user devices 501 are examples and are not intended to belimiting. Examples of the user devices 501 include, but are not limitedto, a desktop computer 502, a cellular/mobile phone 503, a tablet device504, and a laptop computer 505. It is to be appreciated that the userdevices 501 can include other devices such as, but not limited to, awearable device, a personal digital assistant (PDA), a video gameconsole, a television, or the like. In an embodiment, the user devices501 can alternatively be referred to as client devices 501. In such anembodiment, the client devices 501 can be in communication with theserver device 535 through the network 540. One or more of the clientdevices 501 can be in communication with another of the client devices501 through the network 540 in an embodiment.

The processor 510 can retrieve and execute programming instructionsstored in the memory 520 and/or the storage 530. The processor 510 canalso store and retrieve application data residing in the memory 520. Theinterconnect 550 is used to transmit programming instructions and/orapplication data between the processor 510, the user I/O 515, the memory520, the storage 530, and the network I/O 540. The interconnect 550 canbe, for example, one or more busses or the like. The processor 510 canbe a single processor, multiple processors, or a single processor havingmultiple processing cores. In some embodiments, the processor 510 can bea single-threaded processor. In an embodiment, the processor 510 can bea multi-threaded processor.

The user I/O 515 can include a display 516 and/or an input 517,according to an embodiment. It is to be appreciated that the user I/O515 can be one or more devices connected in communication with thecomputer device 500 that are physically separate from the computerdevice 500. For example, the display 516 and input 517 for the desktopcomputer 502 can be connected in communication but be physicallyseparate from the computer device 500. In some embodiments, the display516 and input 517 can be physically included with the computer device500 for the desktop computer 502. In an embodiment, the user I/O 515 canphysically be part of the user device 501. For example, thecellular/mobile phone 503, the tablet device 504, and the laptop 505include the display 516 and input 517 that are part of the computerdevice 500. The server device 535 generally may not include the user I/O515. In an embodiment, the server device 535 can be connected to thedisplay 516 and input 517.

The display 516 can include any of a variety of display devices suitablefor displaying information to the user. Examples of devices suitable forthe display 516 include, but are not limited to, a cathode ray tube(CRT) monitor, a liquid crystal display (LCD) monitor, a light emittingdiode (LED) monitor, or the like.

The input 517 can include any of a variety of input devices or inputmeans suitable for receiving an input from the user. Examples of devicessuitable for the input 517 include, but are not limited to, a keyboard,a mouse, a trackball, a button, a voice command, a proximity sensor, anocular sensing device for determining an input based on eye movements(e.g., scrolling based on an eye movement), or the like. It is to beappreciated that combinations of the foregoing inputs 517 can beincluded for the user devices 501. In some embodiments the input 517 canbe integrated with the display 516 such that both input and output areperformed by the display 516.

The memory 520 is generally included to be representative of a randomaccess memory such as, but not limited to, Static Random Access Memory(SRAM), Dynamic Random Access Memory (DRAM), or Flash. In someembodiments, the memory 520 can be a volatile memory. In someembodiments, the memory 520 can be a non-volatile memory. In someembodiments, at least a portion of the memory can be virtual memory.

The storage 530 is generally included to be representative of anon-volatile memory such as, but not limited to, a hard disk drive, asolid state device, removable memory cards, optical storage, flashmemory devices, network attached storage (NAS), or connections tostorage area network (SAN) devices, or other similar devices that maystore non-volatile data. In some embodiments, the storage 530 is acomputer readable medium. In some embodiments, the storage 530 caninclude storage that is external to the computer device 500, such as ina cloud.

The network I/O 525 is configured to transmit data via a network 540.The network 540 may alternatively be referred to as the communicationsnetwork 540. Examples of the network 540 include, but are not limitedto, a local area network (LAN), a wide area network (WAN), the Internet,or the like. In some embodiments, the network I/O 525 can transmit datavia the network 540 through a wireless connection using Wi-Fi,Bluetooth, or other similar wireless communication protocols. In someembodiments, the computer device 500 can transmit data via the network540 through a cellular, 3G, 4G, or other wireless protocol. In someembodiments, the network I/O 525 can transmit data via a wire line, anoptical fiber cable, or the like. It is to be appreciated that thenetwork I/O 525 can communicate through the network 540 through suitablecombinations of the preceding wired and wireless communication methods.

The server device 535 is generally representative of a computer device500 that can, for example, respond to requests received via the network540 to provide, for example, data for rendering a website on the userdevices 501. The server 535 can be representative of a data server, anapplication server, an Internet server, or the like.

Aspects described herein can be embodied as a system, method, or acomputer readable medium. In some embodiments, the aspects described canbe implemented in hardware, software (including firmware or the like),or combinations thereof. Some aspects can be implemented in anon-transitory, tangible computer readable medium, including computerreadable instructions for execution by a processor. Any combination ofone or more computer readable medium(s) can be used.

The computer readable medium can include a computer readable signalmedium and/or a computer readable storage medium. A computer readablestorage medium can include any tangible medium capable of storing acomputer program for use by a programmable processor to performfunctions described herein by operating on input data and generating anoutput. A computer program is a set of instructions that can be used,directly or indirectly, in a computer system to perform a certainfunction or determine a certain result. Examples of computer readablestorage media include, but are not limited to, a floppy disk; a harddisk; a random access memory (RAM); a read-only memory (ROM); asemiconductor memory device such as, but not limited to, an erasableprogrammable read-only memory (EPROM), an electrically erasableprogrammable read-only memory (EEPROM), Flash memory, or the like; aportable compact disk read-only memory (CD-ROM); an optical storagedevice; a magnetic storage device; other similar device; or suitablecombinations of the foregoing. A computer readable signal medium caninclude a propagated data signal having computer readable instructions.Examples of propagated signals include, but are not limited to, anoptical propagated signal, an electro-magnetic propagated signal, or thelike. A computer readable signal medium can include any computerreadable medium that is not a computer readable storage medium that canpropagate a computer program for use by a programmable processor toperform functions described herein by operating on input data andgenerating an output.

An embodiment can be provided to an end-user through a cloud-computinginfrastructure. Cloud computing generally includes the provision ofscalable computing resources as a service over a network (e.g., theInternet or the like).

The terminology used in this specification is intended to describeparticular embodiments and is not intended to be limiting. The terms“a,” “an,” and “the” include the plural forms as well, unless clearlyindicated otherwise. The terms “comprises” and/or “comprising,” whenused in this specification, specify the presence of the stated features,integers, steps, operations, elements, and/or components, but do notpreclude the presence or addition of one or more other features,integers, steps, operations, elements, and/or components.

With regard to the preceding description, it is to be understood thatchanges may be made in detail, especially in matters of the constructionmaterials employed and the shape, size, and arrangement of parts withoutdeparting from the scope of the present disclosure. This specificationand the embodiments described are exemplary only, with the true scopeand spirit of the disclosure being indicated by the claims that follow.

1-16. (canceled)
 17. A system, comprising: a server having a processorand a memory; a communication network; and a database in electroniccommunication with the server via the communication network, wherein theserver includes: an electronic product recognizer that receives asegmented image from a camera of an electronic mobile device, thesegmented image including a plurality of segments; performs an imagerecognition using each of the plurality of segments to identify theproduct in each of the plurality of segments; and outputs one or morerecognized products identified in the image recognition.
 18. The systemof claim 17, wherein the server is stored on premises of a retail store.19. The system of claim 17, wherein the electronic product recognizerperforms the image recognition in less than 2 seconds.
 20. The system ofclaim 17, further comprising a retail store, wherein the server isstored on premises of the retail store, and the communication network isa local area network of the retail store.
 21. The system of claim 17,wherein the electronic product recognizer performs the image recognitionwith a greater than 95% confidence.
 22. The system of claim 17, whereinthe image recognition includes a gross filtering approach and a finefiltering approach.
 23. A computer-implemented method to electronicallyrecognize a product in an electronic image captured via an electronicmobile device, the method comprising: receiving, by a server, asegmented image from a camera of the electronic mobile device, the videostream including a plurality of segments; performing an imagerecognition using each of the plurality of segments to identify theproduct in each of the plurality of segments; and outputting, from theserver, one or more recognized products identified in the imagerecognition.
 24. The computer-implemented method of claim 23, whereinthe performing the image recognition includes applying a gross filteringapproach and a fine filtering approach.
 25. The computer-implementedmethod of claim 24, wherein the gross filtering approach includes one ormore of a location determination, a logo determination, and a branddetermination.
 26. The computer-implemented method of claim 25, whereinthe location determination includes determination of a current locationbased on a global positioning system (GPS) sensor on the electronicmobile device, a previous location based on the GPS sensor on theelectronic mobile device, an in-store positioning system (IPS), and aplanogram of the retail store.
 27. The computer-implemented method ofclaim 24, wherein the fine filtering approach includes one or more of ashelf edge determination, a user path determination, an OCR, a brandmarker determination, and a color determination.
 28. Thecomputer-implemented method of claim 24, further comprising selecting amatch from a pool of possible products determined from the productsidentified in the fine filtering approach.
 29. The computer-implementedmethod of claim 28, wherein selecting the match includes OCR anddetermination of an item marker.
 30. The computer-implemented method ofclaim 23, wherein the method is completed in less than at or about 2seconds.
 31. The computer-implemented method of claim 23, wherein themethod has an confidence greater than at or about 95%.